Assessing Model Adequacy

Environmental models are simplified representations of complex physical systems. The implementation of any such model, as a computer simulator, involves further simplifications and approximations. The value of the resulting simulator, in giving scientific and practical insights into the functioning of the corresponding physical system, depends both on the nature and degree of these simplifications and also on the objectives for which the model is to be used. This chapter provides an introduction to some basic general techniques for assessing the adequacy of a computer model for its intended purpose. There are many ways to approach this question. We will take the view that the aim of the model is to provide some, necessarily partial, information about the behaviour of the system, and we will consider the model adequate for an intended task if the information that is provided by the simulator is sufficient to allow us to carry out this task. We would usually prefer precise forecasts of system behaviour but we may often be able to tolerate probabilistic forecasts, provided that we are able to quantify the level of uncertainty with which these forecasts should be interpreted, and to confirm that this uncertainty is not so large as to prevent us from achieving our objectives. This will, inevitably, be a pragmatic judgement. Therefore, in our account, we will outline some basic methods for assessing the degree of uncertainty that it would be reasonable to associate with model outcomes. It is beyond the scope of this account to produce precise quantifications of predictive uncertainty, as such analysis requires rather more technical machinery than we have space to describe. Instead, we offer some basic tools for making order of magnitude quantifications for such uncertainties, which should indicate whether the limitations of the model are likely to render it unfit for the task at hand. This is by no means a complete account, even for our stated goal, as such analysis is strongly dependent both on the scientific context and also on the size and complexity of the model. In the next section, we outline the general methods that we suggest and, in the following sections, we illustrate how the methods can be used in practice, by applying them to a rainfall runoff model.